-
- SuperBorracho
18 Oct 2021 18:12
Hello specialists!
This thread is an announcement of several things that will come soon to this forums:
First, i've just made the first autoplay decompiler script. This means that any autoplay app can be fully reverted back to autoplay project or obtain any kind of info from the source.
You may think that i've already did that with project dumper tool, but this works in totally different way, parsing the _proj.dat file structure statically, instead loading the code and injecting dumping api stubs dynamically.
As this is going to be a bit long post, ill write more about this in the next lines...
The other announcement, is that from next year we're renewing the forums, and id like to get some feedback from community in what you would do to make this better place. I'll reserve my own ideas now because id like to hear from people without biasing, but i can advance, we're going to have an online tool for all kinds of AMS manipulation (cdd password extraction, sfx extraction, _proj.dat reverser etc...) and probably a copy of the messages of this forums will be saved for historical propourses, but many thousands of dead links and useless walls of text will have to be archived in order to keep sanity and cleanlyness.
So for the 10th anniversary of AMSSpecialist were moving forward, releasing the definitive autoplay dumping tool, and starting new horizons... with our forums, and development.
The next milestone for amsspecialist is to start an open source clone of autoplay media studio. Id also gladly hear feedback on this, since theres a lot possibilities like solving old known problems but breaking compatibility with original ams. So it will be an interesting project for the community and to rebuild this community around better project. But that will come after were done with ams, ofcourse.
So, lets talk about AMS decompiler in full lenght:
It all started while i was looking at a tool called "kaitai struct", i wanted to learn using it, and real learning comes with practice, so i remembered the only data structure that ive never been able to unpack and given i have some knoweledge of it, would be an easy pick to practice with... the autoplay's _proj.dat.
In this post, ill try to describe details of the process and interesting stuff from this undocumented file.
To begin, id need to easily get _proj.dat files with small modifications quickly to be able of compare differences. usually, you generate an executable, get its password, then dump the _proj.dat from .cdd file, this was unaceptable for this proporuse, so ive prepared a msimg.dll hook that just writes the _proj.dat file contents to a file on the harddisk without any other protection.
Then, ive noticed the saving routine of AutoPlayDesign.exe is very similar to dump the _proj.dat and to save the .autoplay project, so it be more or less safe to assume most of the variables would appear in the same order on both files. This speed up the process by magnitudes as then only the type is unknown, and most of them are usually 4 byte integers.
To be clear, im working with ams 8.5.x.x demo and professional and now im in the process of reversing diffs with older executable versions, this is why tool is not out yet, since now i have a path of downstream changes until ams 6.0.0.0 wich is the last version im up to support but i still have to look at 5... also only cdd extraction is done yet to 8.5.x.x, so now im on the process of documenting older versions... maybe even some could help with that! dont heystate to offer your help...
The _proj.dat file is the product of MFC class serialization, via CArchive class using CSerializable and CObList derived objects, and also some few ATL data structures like maps and vectors, which are predeccesors of now popular std c++ extensions. All of this makes this task a bit harder since this libs are closed src from microsoft, but its simplicity of operation can help to reverse its format without looking so deep.
From a static analisys of the autoplay's executable, proj.dat seems to be readed in blocks, depending if there are some bytes in it at specific relative position. So we're not reading a serialized object, but a bunch of them. MFC serialization seems to require a short value (2 byte) indicating the class type to parse, and loading it for a first time using an optional string. This is true for the most part of the file, but many initial sections lack of this since the type is described on the runtime code.
Many values are unknown since they probably are autogenerated fill values, but may also contain data, so further reversing should be done on unknown values.
Order of the PE:
-Startup code, a blob containing _error_messages.lua + _notification_messages.lua + database init code
-AMS runtime ID: a series of values that identify the compiling ams runtime version, product and unix timestamp of compilation
-Window style: the global preferences, width, height, titlebar etc...
-Window options, style, auto bg music, transparent window, intro video... a bunch of properties with all the project settings.
-Globals code
-Clicksound and other defaults, like startup page
-Action plugin list
-menu bar
-Global coding events (On startup, on shutdown....)
-cjstyles: custom styles (probably only in ams 8)
-Pages (containing a subarray of data, events and objects)
-Dialogs (very similar to pages)
And thats it! obviosly it gets more complicated while going deep into objects, as they all share some strcuture but they have a custom property struct for each type of object.
Here's a graphviz visualization where you can understand the magnitude of the task of decompiling this file, given that for it to correctly work there must be 0 (zero) errors on the data or the reversing process will fail miserably, and its must be described in full
This work has been done in 2 days, one of them being wasted in the default data of the tree object, that was interestingly hard to reverse.
Sample decompiled to json: https://pastebin.com/E0fbAstE
and sample autoplay project: https://pastebin.com/HRzPWiR8
How to test a decompilation of 8.5.x.x proj.dat, go to ide.kaitai.io and paste my decompiler script, drop your _proj.dat file to the ide and you will get it either totally parsed or catastophic error will happen. im interested into the problematic ones. My aim is to fully test the format then release a converter to ams project or apz. As i have the installer for older versions, ill need to get some time to go back installing them and dumping the cdd breaking + pe changes for all of them, then will be the time to revert back to project.
This thread is an announcement of several things that will come soon to this forums:
First, i've just made the first autoplay decompiler script. This means that any autoplay app can be fully reverted back to autoplay project or obtain any kind of info from the source.
You may think that i've already did that with project dumper tool, but this works in totally different way, parsing the _proj.dat file structure statically, instead loading the code and injecting dumping api stubs dynamically.
As this is going to be a bit long post, ill write more about this in the next lines...
The other announcement, is that from next year we're renewing the forums, and id like to get some feedback from community in what you would do to make this better place. I'll reserve my own ideas now because id like to hear from people without biasing, but i can advance, we're going to have an online tool for all kinds of AMS manipulation (cdd password extraction, sfx extraction, _proj.dat reverser etc...) and probably a copy of the messages of this forums will be saved for historical propourses, but many thousands of dead links and useless walls of text will have to be archived in order to keep sanity and cleanlyness.
So for the 10th anniversary of AMSSpecialist were moving forward, releasing the definitive autoplay dumping tool, and starting new horizons... with our forums, and development.
The next milestone for amsspecialist is to start an open source clone of autoplay media studio. Id also gladly hear feedback on this, since theres a lot possibilities like solving old known problems but breaking compatibility with original ams. So it will be an interesting project for the community and to rebuild this community around better project. But that will come after were done with ams, ofcourse.
So, lets talk about AMS decompiler in full lenght:
It all started while i was looking at a tool called "kaitai struct", i wanted to learn using it, and real learning comes with practice, so i remembered the only data structure that ive never been able to unpack and given i have some knoweledge of it, would be an easy pick to practice with... the autoplay's _proj.dat.
In this post, ill try to describe details of the process and interesting stuff from this undocumented file.
To begin, id need to easily get _proj.dat files with small modifications quickly to be able of compare differences. usually, you generate an executable, get its password, then dump the _proj.dat from .cdd file, this was unaceptable for this proporuse, so ive prepared a msimg.dll hook that just writes the _proj.dat file contents to a file on the harddisk without any other protection.
Then, ive noticed the saving routine of AutoPlayDesign.exe is very similar to dump the _proj.dat and to save the .autoplay project, so it be more or less safe to assume most of the variables would appear in the same order on both files. This speed up the process by magnitudes as then only the type is unknown, and most of them are usually 4 byte integers.
To be clear, im working with ams 8.5.x.x demo and professional and now im in the process of reversing diffs with older executable versions, this is why tool is not out yet, since now i have a path of downstream changes until ams 6.0.0.0 wich is the last version im up to support but i still have to look at 5... also only cdd extraction is done yet to 8.5.x.x, so now im on the process of documenting older versions... maybe even some could help with that! dont heystate to offer your help...
The _proj.dat file is the product of MFC class serialization, via CArchive class using CSerializable and CObList derived objects, and also some few ATL data structures like maps and vectors, which are predeccesors of now popular std c++ extensions. All of this makes this task a bit harder since this libs are closed src from microsoft, but its simplicity of operation can help to reverse its format without looking so deep.
From a static analisys of the autoplay's executable, proj.dat seems to be readed in blocks, depending if there are some bytes in it at specific relative position. So we're not reading a serialized object, but a bunch of them. MFC serialization seems to require a short value (2 byte) indicating the class type to parse, and loading it for a first time using an optional string. This is true for the most part of the file, but many initial sections lack of this since the type is described on the runtime code.
Many values are unknown since they probably are autogenerated fill values, but may also contain data, so further reversing should be done on unknown values.
Order of the PE:
-Startup code, a blob containing _error_messages.lua + _notification_messages.lua + database init code
-AMS runtime ID: a series of values that identify the compiling ams runtime version, product and unix timestamp of compilation
-Window style: the global preferences, width, height, titlebar etc...
-Window options, style, auto bg music, transparent window, intro video... a bunch of properties with all the project settings.
-Globals code
-Clicksound and other defaults, like startup page
-Action plugin list
-menu bar
-Global coding events (On startup, on shutdown....)
-cjstyles: custom styles (probably only in ams 8)
-Pages (containing a subarray of data, events and objects)
-Dialogs (very similar to pages)
And thats it! obviosly it gets more complicated while going deep into objects, as they all share some strcuture but they have a custom property struct for each type of object.
Here's a graphviz visualization where you can understand the magnitude of the task of decompiling this file, given that for it to correctly work there must be 0 (zero) errors on the data or the reversing process will fail miserably, and its must be described in full
This work has been done in 2 days, one of them being wasted in the default data of the tree object, that was interestingly hard to reverse.
Sample decompiled to json: https://pastebin.com/E0fbAstE
and sample autoplay project: https://pastebin.com/HRzPWiR8
How to test a decompilation of 8.5.x.x proj.dat, go to ide.kaitai.io and paste my decompiler script, drop your _proj.dat file to the ide and you will get it either totally parsed or catastophic error will happen. im interested into the problematic ones. My aim is to fully test the format then release a converter to ams project or apz. As i have the installer for older versions, ill need to get some time to go back installing them and dumping the cdd breaking + pe changes for all of them, then will be the time to revert back to project.
