[Bugs] Moth Lamp

[simplex]

I don't see anything wrong with it, since Ipsquiggle and Cheerio seem to be absent (also, the mod tools never made it out of alpha?), and we don't know who to ask about it. Of course, we could always just kick stuff around until we figure it out.

The last time I asked JoeW to summon someone was Cheerio when I ported the mod tools. Cheerio replied shortly after, saying he'd take a look once he made it back from vacation, and that was the last I heard on the matter.

I don't think we'll figure out how to extend the scml compiler without further insight on how the bounding boxes should be represented in the animation files.

[debugman18]

The last time I asked JoeW to summon someone was Cheerio when I ported the mod tools. Cheerio replied shortly after, saying he'd take a look once he made it back from vacation, and that was the last I heard on the matter.

I don't think we'll figure out how to extend the scml compiler without further insight on how the bounding boxes should be represented in the animation files.

 

Does Cheerio's creature animation tutorials have the scml files? I would assume it has a proper bounding box. Maybe there is a noticeable difference between it and ours?

[simplex]

Does Cheerio's creature animation tutorials have the scml files? I would assume it has a proper bounding box. Maybe there is a noticeable difference between it and ours?

His scml files should have the same issue. It's not about the scml, but the intermediate xml the scml converter generates and then passes along to the Python scripts.

Calculating the bounding box properly was in his TODO list for the mod tools (as you can see in the README.md of his github repo for them), and he told Malacath this would be the next thing he'd tackle after the Linux/Mac porting were done. But then he disappeared, even before the porting had been completed (in fact, almost nothing of it was done under his watch).

[debugman18]

His scml files should have the same issue. It's not about the scml, but the intermediate xml the scml converter generates and then passes along to the Python scripts.

Calculating the bounding box properly was in his TODO list for the mod tools (as you can see in the README.md of his github repo for them), and he told Malacath this would be the next thing he'd tackle after the Linux/Mac porting were done. But then he disappeared, even before the porting had been completed .

 

Wow, that... just sucks. Well, hopefully a developer will enlighten us, then.

 

Did you see the suggestion in my edit? It's probably not likely to work, but it can't hurt to bring it up.

[simplex]

Would making a component to attach an invisible vanilla asset to a given converted animation work? Doesn't the balloon hound's balloon's clickbox act as part of the hound's?

What do you mean by attach a vanilla asset? Tag another entity along with an invisible atlas? Because if so it doesn't really work. I tried something like this in the snailking mod, but the issue is the game will then reference that entity and its components (such as when picking and applying an action).

And the balloon hound is a hound. I just attached a balloon to it as a child entity (and made the balloon unclickable).

[debugman18]

What do you mean by attach a vanilla asset? Tag another entity along with an invisible atlas? Because if so it doesn't really work. I tried something like this in the snailking mod, but the issue is the game will then reference that entity and its components (such as when picking and applying an action).

And the balloon hound is a hound. I just attached a balloon to it as a child entity (and made the balloon unclickable).

 

Yes, to tag another entity along to it.

 

So it doesn't work then, I figured. I'm reading the scml mod tools code, maybe there's something there that'll help. I'm curious how the dragonblood tree got such a large bounding box, while nearly all of the others have tiny ones.

 

[simplex]

So it doesn't work then, I figured. I'm reading the scml mod tools code, maybe there's something there that'll help. I'm curious how the dragonblood tree got such a large bounding box, while nearly all of the others have tiny ones.

Do you know C++?

(well, the scml converter is barely C++, it's essentially C; I'd honestly rather write a new scml converter in a more object-oriented fashion than to tweak the existing one)

[debugman18]

Do you know C++?

(well, the scml converter is barely C++, it's essentially C; I'd honestly rather write a new scml converter in a more object-oriented fashion than to tweak the existing one)

 

I don't know C or C++. Although I'm reading up on some of the operators right now. I may not come out of it with a solution (it's pretty unlikely) but it's helpful to read code.

 

 

[simplex]

I don't know C or C++. Although I'm reading up on some of the operators right now. I may not come out of it with a solution (it's pretty unlikely) but it's helpful to read code.

The syntax and basic types are fairly similar to Java. But it's certainly not Java .

Similarity between languages goes a long way, though. I may not know C#, but that didn't stop me from learning how Matt's tools worked by reading their code (he was very helpful in not only writing his tools but also keeping a documentation of his discoveries on the TEX format in tables and such, though this was much more limited than what went into the code itself).

[debugman18]

The syntax and basic types are fairly similar to Java. But it's certainly not Java .

Similarity between languages goes a long way, though. I may not know C#, but that didn't stop me from learning how Matt's tools worked by reading their code (he was very helpful in not only writing his tools but also keeping a documentation of his discoveries on the TEX format in tables and such, though this was much more limited than what went into the code itself).

 

I noticed a couple of similarities.

 

It's difficult to read and understand it, but I can grasp the more basic utilities of it.

 

I read in the other thread that squeek had found that the bounding box was related to the position and pivot of the animation in Spriter. 

 

I took a look at the character scml's on Github and noticed that our mainline blocks are different from that one, as well as the variables of abs_x and abs_y. In the wilson build, abs_x and abs_y are 0, while in ours the abs_x and abs_y are various different values.

 

Maybe I'm going in the completely wrong direction here, but I'll test some conversions tomorrow and see if I can narrow it down to any particular value.

[simplex]

@debugman18

So, I wrote a little script to try an idea out.

klickable.zip

The position of the boxes is still off, but the size looks about right (and the clicking area is greatly improved). I replaced the cloud_algae and beanlet anims in our anim/ folder with the tweaked ones (keeping the old ones with an "old_" prefix), and as you can see the bounding box seems to track the animation motion correctly as well (of the beanlet, of course, since the cloud algae doesn't tend to move much ).

EDIT: Waaaait a minute, I just noticed something in my script that might explain why the position offset of the boxes seems off: I forgot to add the offset I'm computing .

I'll fix it and post the new result later.

EDIT 2: There, much better:

I also updated the attached zip with the script.

[debugman18]

@debugman18

So, I wrote a little script to try an idea out.

klickable.zip

The position of the boxes is still off, but the size looks about right (and the clicking area is greatly improved). I replaced the cloud_algae and beanlet anims in our anim/ folder with the tweaked ones (keeping the old ones with an "old_" prefix), and as you can see the bounding box seems to track the animation motion correctly as well (of the beanlet, of course, since the cloud algae doesn't tend to move much ).

EDIT: Waaaait a minute, I just noticed something in my script that might explain why the position offset of the boxes seems off: I forgot to add the offset I'm computing .

I'll fix it and post the new result later.

EDIT 2: There, much better:

I also updated the attached zip with the script.

 

I did not expect a fix yet.

 

I'm not understanding what's happening in the script. Apparently the height, width, and x and y positions weren't being translated appropriately in the mod tools?

 

Oh, and it seems to be that the octocopter scml won't export for me.It gives me a segmentation fault. -.- Other things will, just not it. I managed to get your script working (I'm supposed to use the build.xml and animation.xml right?) but that doesn't do me much good if I can't export things with the mod tools.

 

Edit: So, something happened to the exported images somewhere along the line, because reverting to an earlier commit (a couple of weeks back almost, I think) allowed me to convert the octocopter stuff. (This was on windows, I haven't tested that on Linux) However, the hitbox didn't change. Could you elaborate on the steps to reconvert the scml properly?

[simplex]

I did not expect a fix yet.

 

I'm not understanding what's happening in the script. Apparently the height, width, and x and y positions weren't being translated appropriately in the mod tools?

They weren't being calculated. The scml converter was just taking the position and dimensions of the first animation symbol it found and saying "ok, that's the size of the whole thing" . It weren't even considering the rotations/translations/scalings/etc. of the symbol, just using the raw symbol size.

What I'm doing is calculate the proper bounding box. In each frame, I take the size of each animation symbol appearing in the frame and, combining that with the transformation matrix and translation applied to it in the frame I compute the minimum and maximum x and y coordinates for it. Iterating through all symbols appearing in the frame, I get the absolute minimum and maximum x and y coordinates of the whole frame. Then I update the "x", "y", "w" and "h" attributes of the frame (the first two being the minimum x and y coordinates and the latter two being the differences of the maximum with the minimum).

To be honest, I had a hunch this is how it should be done ever since I ported the tools (that's why the idea came to me so quickly). I planned on testing it and if it worked patching the scml converter, but after the port was just ignored I didn't feel like wasting any more time and left it alone. Unless I got explicit confirmation from a dev on how this worked, I didn't plan on spending any time on it. But then you started messing with it, so our team was already spending time on it, and I decided to just try it anyway.

 

Edit: So, something happened to the exported images somewhere along the line, because reverting to an earlier commit (a couple of weeks back almost, I think) allowed me to convert the octocopter stuff. (This was on windows, I haven't tested that on Linux) However, the hitbox didn't change. Could you elaborate on the steps to reconvert the scml properly?

It's quite hard to use right now, because you need to first run the scml compiler (or the autocompiler), unzip the zip in exported/, replace animation.xml with the script's output, rezip and then use the export.py script of the mod tools directly on it, passing the right parameters.

I wrote it as a quick proof of concept script just to see if my idea worked. Even though I don't welcome the prospect of working on the very C-like code of the scml converter, I think embedding the fix right there is best.

What's your workflow with the mod tools, but the way? Do you keep them in a folder by dont_starve and use the autocompiler, just use the scml compiler directly, or what?

[debugman18]

They weren't being calculated. The scml converter was just taking the position and dimensions of the first animation symbol it found and saying "ok, that's the size of the whole thing throughout all animations" .

What I'm doing is calculate the proper bounding box. In each frame, I take the size of each animation symbol appearing in the frame and, combining that with the transformation matrix and translation applied to it in the frame I compute the minimum and maximum x and y coordinates for it. Iterating through all symbols appearing in the frame, I get the absolute minimum and maximum x and y coordinates of the whole frame. Then I update the "x", "y", "w" and "h" attributes of the frame (the first two being the minimum x and y coordinates and the latter two being the differences of the maximum with the minimum).

To be honest, I had a hunch this is how it should be done ever since I ported the tools. I planned on testing it and if it worked patching the scml converter, but after the port was just ignored I didn't feel like wasting any more time and left it alone. So I simply revisited that idea now .

It's quite hard to use right now, because you need to first run the scml compiler (or the autocompiler), unzip the zip in exported/, replace animation.xml with the script's output, rezip and then use the export.py script of the mod tools directly on it, passing the right parameters.

I wrote it as a quick proof of concept script just to see if my idea worked. Even though I don't welcome the prospect of working on the very C-like code of the scml converter, I think embedding the fix right there is best.

What's your workflow with the mod tools, but the way? Do you keep them in a folder by dont_starve and use the autocompiler, just use the scml compiler directly, or what?

 

I use the scml compiler directly.

[simplex]

I use the scml compiler directly.

Great. I'd rather not have the png compiler messing with our Makefile-based image compilation.

[simplex]

The scml compiler patch should be ready soon-ish.

But geez, is it hard to extend that code even a bit. There's no encapsulation, everything is stored as a raw array. It's like I'm back in my days of programming Assembly. Even properties of animation symbols' frames: instead of grouping the frames by symbol, all frames are put together in the same array, and then there's one array mapping a symbol index (its "declaration position" in the file) to the first corresponding index in the frames array, and another mapping a symbol index to its number of frames. So you have to check both to figure out the "slice" of the frames array you're interested in. Everything is flat with no abstraction, so you end up with function calls like this:

        convert_anim_timelines_to_frames(            anim_lengths[i],            anim_loopings[i],            anim_timeline_counts[i],            &anim_timeline_key_start_indices[timeline_start_index],            &anim_timeline_key_counts[timeline_start_index],            timeline_key_times,            timeline_key_positions,            timeline_key_dimensions,            timeline_key_pivots,            timeline_key_scales,            timeline_key_angles,            timeline_key_z_indices,            timeline_key_spins,            timeline_key_symbol_frame_nums,            timeline_key_names,            timeline_key_parent_ids,            timeline_key_alphas,            anim_frame_counts[i],                 &frame_element_counts[frame_element_count_index],            &frame_element_start_indices[frame_element_count_index],            &frame_indices[frame_element_count_index],            &frame_positions[frame_element_count_index],            &frame_dimensions[frame_element_count_index],            timeline_frame_symbol_frame_nums,            timeline_frame_names,            timeline_frame_layer_names,            timeline_frame_positions,            timeline_frame_dimensions,            timeline_frame_pivots,            timeline_frame_scales,            timeline_frame_angles,            timeline_frame_z_indices,            timeline_frame_parent_ids,            timeline_frame_alphas,            element_index            );

It took me 20 minutes to implement the fix in Lua, but a hell more than that just to track the spaghetti references and figure out where what is used and when, so that I may start to modify the code...

[simplex]

@debugman18

The scml compiler has been patched to compute bounding boxes properly, and all of our custom builds/animations have been recompiled.

I also made the mod tools a bit less hardcoded by allowing some customisation via environment variables (as well as adding a "-f" force option to the scml compiler to recreate the build regardless of timestamps). Using these, I integrated scml compilation in our Make-based build chain. So now

$ make

will compile custom builds as well if they need updating. Typing "make exported" will take care of just the custom builds, and "make -B exported" will force recompilation of all custom builds (a "make -B" still recompiles everything, which now includes custom builds).

You just need to create a file config.mk in the UpAndAway/make/ folder specifying the location of the scml compiler in your system (by exporting a SCML_COMPILER variable). Mine consists of

export SCML_COMPILER=$(HOME)/Documents/utils/dontstarve/ds_mod_tools/build/linux/mod_tools/scml

where as you can see $(HOME) can be used to specify your home directory. The make/config.mk path has been in our .gitignore for some time, so don't worry about it being added to the repo.

Furthermore, when using the scml compiler via our Makefiles, two things are done differently (via the environment variable setting I mentioned previously):

• The scml log is put in UpAndAway/log/scml_log.txt.
• The temporary files created by the scml compilation are put in a proper temporary directory in /tmp, which is erased after the scml compiler exits (this is true even if you abort it via Ctrl-C, kill or killall). The name of the temporary directory is guaranteed to be unique, not clashing with existing directories, since it is generated by the command "mktemp -d" (the creation and deletion of the temp directory is actually handled by the wrapper script tools/scml_wrapper.sh).

When recompiling all of our anims, I noticed several pngs had issues which prevented compilation. Just like in the beanlet_zealot case, this was caused by them not being proper RGB(A) images (some were indexed, some were grayscale). So I also added a new Make target: entering

$ make fiximages

will fix all pngs in exported/.

However, the recompiled antlion build is completely wrong. Not just the bounding box, everything is wrong . But I don't think the bounding box changes were what caused it, maybe it had been compiled before the scml compiler got bone support, so that's why it's much more wrong now? Anyway, all other builds I've checked (which are most of them) are correct, and with the correct bounding boxes. I couldn't recompile the dragonblood_tree build, though, since it's not in our exported/ directory.

---

EDIT:

Do you know if @Malacath posted the development version of Waverly somewhere public? I'd like to fix the bounding box of the weeping willow now that the mod tools compute it properly.

[KidneyBeanBoy]

EDIT:

Do you know if @Malacath posted the development version of Waverly somewhere public? I'd like to fix the bounding box of the weeping willow now that the mod tools compute it properly.

Check it's thread in the submissions subforum. I think he posted it there.

[simplex]

Check it's thread in the submissions subforum. I think he posted it there.

I went through several (but not all) pages, but I couldn't find it. All I could find was the link to the mod upload and several broken links to his dropbox account.

[KidneyBeanBoy]

I went through several (but not all) pages, but I couldn't find it. All I could find was the link to the mod upload and several broken links to his dropbox account.

Oh. So is this still broken?:

You can have the whole mod of course ^^ Original assets included.

[simplex]

Oh. So is this still broken?:

Yes, if you follow the link you'll see it gives an error 404.

[debugman18]

The scml compiler patch should be ready soon-ish.

But geez, is it hard to extend that code even a bit. There's no encapsulation, everything is stored as a raw array. It's like I'm back in my days of programming Assembly. Even properties of animation symbols' frames: instead of grouping the frames by symbol, all frames are put together in the same array, and then there's one array mapping a symbol index (its "declaration position" in the file) to the first corresponding index in the frames array, and another mapping a symbol index to its number of frames. So you have to check both to figure out the "slice" of the frames array you're interested in. Everything is flat with no abstraction, so you end up with function calls like this:

        convert_anim_timelines_to_frames(            anim_lengths[i],            anim_loopings[i],            anim_timeline_counts[i],            &anim_timeline_key_start_indices[timeline_start_index],            &anim_timeline_key_counts[timeline_start_index],            timeline_key_times,            timeline_key_positions,            timeline_key_dimensions,            timeline_key_pivots,            timeline_key_scales,            timeline_key_angles,            timeline_key_z_indices,            timeline_key_spins,            timeline_key_symbol_frame_nums,            timeline_key_names,            timeline_key_parent_ids,            timeline_key_alphas,            anim_frame_counts[i],                 &frame_element_counts[frame_element_count_index],            &frame_element_start_indices[frame_element_count_index],            &frame_indices[frame_element_count_index],            &frame_positions[frame_element_count_index],            &frame_dimensions[frame_element_count_index],            timeline_frame_symbol_frame_nums,            timeline_frame_names,            timeline_frame_layer_names,            timeline_frame_positions,            timeline_frame_dimensions,            timeline_frame_pivots,            timeline_frame_scales,            timeline_frame_angles,            timeline_frame_z_indices,            timeline_frame_parent_ids,            timeline_frame_alphas,            element_index            );

It took me 20 minutes to implement the fix in Lua, but a hell more than that just to track the spaghetti references and figure out where what is used and when, so that I may start to modify the code...

 

 

Holy crap that function...

 

More importantly, holy crap the bounding box bug has been solved.

 

Also, Assembly? I commend you, I'd rather learn to code in C# in a foreign language than even touch Assembly.

[simplex]

Also, Assembly? I commend you, I'd rather learn to code in C# in a foreign language than even touch Assembly.

Yeah, it was the third "language" (it's hard to call it a language because it depends on the processor's architecture, and besides there are two different syntaxes for it, Intel's and AT&T's) I learned, after C and Lua.

Back when I was first learning how to program I had a strong interest in low level stuff, dealing directly with how the computer actually computes. I had a strong change of view on that, though, and nowadays I'm much more about higher degrees of abstraction, code readability and flexibility and other high level goodies instead of bit scrubbing at the low level. But I still value my Assembly/C experience as knowledge, since it's useful to know what's really going on behind the scenes while you're working at a higher level of abstraction.

Anyway, I've been thinking (for quite some time): do you think it would be substantially useful for U&A if I wrote a build/animation decompiler, which turns the game's anim.bin/build.bin/atlas-0.tex into a Spriter project?

[debugman18]

Yeah, it was the third "language" (it's hard to call it a language because it depends on the processor's architecture, and besides there are two different syntaxes for it, Intel's and AT&T's) I learned, after C and Lua.

Back when I was first learning how to program I had a strong interest in low level stuff, dealing directly with how the computer actually computes. I had a strong change of view on that, though, and nowadays I'm much more about higher degrees of abstraction, code readability and flexibility and other high level goodies instead of bit scrubbing at the low level. But I still value my Assembly/C experience as knowledge, since it's useful to know what's really going on behind the scenes while you're working at a higher level of abstraction.

Anyway, I've been thinking (for quite some time): do you think it would be substantially useful for U&A if I wrote a build/animation decompiler, which turns the game's anim.bin/build.bin/atlas-0.tex into a Spriter project?

 

That sounds like a lot of work for you to do, but the uses would be pretty awesome. We could edit existing animations (for instance we could do various things with the shopkeeper without building him from scratch.) We could probably make further use from atlases, since we'd be able to edit the animations. (So for instance, for walls, we could make the bounding area for each sprite larger, so we wouldn't have the clipping uses when overwriting vanilla atlases.)

 

We could also add new player animations. There's endless possibilities assuming we can decompile and modify vanilla animations.

 

I could see the uses. It's up to you, though. I'm sure it'd be beneficial to other modders too. It'd be like Matt's tools, except much more useful.

[simplex]

I could see the uses. It's up to you, though. I'm sure it'd be beneficial to other modders too. It'd be like Matt's tools, except much more useful.

The thing is I don't know how the modding community around here will fare even in the short term. After DS stopped being updated, it went quite stale. And now RoG is almost done. That's the main reason why I haven't written it so far.

That sounds like a lot of work for you to do, but the uses would be pretty awesome. We could edit existing animations (for instance we could do various things with the shopkeeper without building him from scratch.) We could probably make further use from atlases, since we'd be able to edit the animations. (So for instance, for walls, we could make the bounding area for each sprite larger, so we wouldn't have the clipping uses when overwriting vanilla atlases.)

 

We could also add new player animations. There's endless possibilities assuming we can decompile and modify vanilla animations.

I'm leaning towards writing it. But if I do tackle this, I won't have much (if at all) time U&A development until it's done. I need to at least fix the whirwind freeze first, but besides that, are there any pressing matters I should take care of?